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CI, 1 Abstract 

<D : 

The existence of fourth family follows from the basics of the Standard Model. Because of the high 
masses of the fourth family quarks, their anomalous decays could be dominant, if certain criteria 
are met. This will drastically change the search strategy at hadron colliders. We show that the 

m ' 

lO ■ fourth SM family down quarks with masses up to 400 — 450 GeV can be observed (or excluded) via 



anomalous decays by Tevatron before the LHC. 
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Recently, the search for the 
experimental HEP community 



burth SM family has drawn attention of both theoretical and 

in 

1H3|. The existence of the fourth SM family is the outcome of 



;he flavor democracy hypothesis and the actual spectrum of the third family fermion masses 
4j-l6j (see, also review [7] and ref's therein). Twenty years' ongoing discussions on "precision 
data vs fourth SM family" have been concluded in favor of the fourth SM family (see 8|, |9fl 
and ref's therein). Today, the mass and the mixing patterns of quarks and leptons are the 
most mysterious aspects of particle physics. Within the SM all these masses and mixings 
are put by hand and constitute the basic source of parameter inflation. The discovery of 
the fourth SM family could provide some systematics for SM fermion masses and mixings, 
which seems chaotic in the three family case. 

Let us remind that flavour physics met a lot of surprises. The first example was discovery 
of //-meson (We were looking for 7r-meson predicted by Yukawa but discovered the "heavy 
electron"). The next example was represented by strange particles (later we understood 
that they contains strange quarks). The story was followed by r-lepton, c- and b-quarks 
discovered in 1970's. Actually, c-quark was foreseen by GIM mechanism and quark-lepton 
symmetry and its mass was estimated in the few GeV region, whereas the discovery of r- 
lepton and b-quark was completely surprising for physicists. According to the Standard 
Model they are the members of the third fermion family, which was completed by the 
discovery of t-quark in 1995 at Tevatron. Actually, we need at least three fermion families 
in order to handle CP- violation within the SM |10| . CP violation is necessary for the 
explanation of Barion Asymmetry of the Universe (BAU). Unfortunately, SM with three 
fermion families does not provide actual magnitude for BAU. Fortunately, the fourth SM 
family could provide additional factor of order of 10 10 and, therefore, solve the problem |ll| . 

The existence of the fourth SM family could affect the Tevatron physics search program 
in two ways. First, it leads to essential enhancement of the Higgs production via gluon 



fusion (see, 



12 



13| and ref's therein). As a result, while in the case of three SM families 



(SM3)Tevatron data on gg ->■ H -> WW -* ll' P^ is process excludes 162 GeV< M# <166 
GeV |lj| at 95% CL, the excluded region becomes 131 GeV < M H < 204 GeV [lj|, if the 
Nature prefers four SM families case. Second, if the masses of the fourth family quarks are 
not so large, they could be directly observed by Tevatron. 

Up to now, all searches for the fourth family quarks have been done by assuming that 
SM decay modes are dominant. Current lower limits on the fourth SM family quark masses 



18 



from direct searches at the Tevatron are: 

a) Md A > 338 GeV at 95% CL coming from the search for new bottom-like quark pair 
decays in same-charged lepton events with an integrated luminosity of 2.7 fb _1 [16], 

b) M UA > 256 GeV at 95% CL coming from the search for heavy top-like quarks, using 
lepton plus jets events with an integrated luminosity 0.76 fb _1 17). 

The heaviness of the t-quark has induced searches for its anomalous interactions 
22j. For the same reason - new quarks are heavier than t-quark - in the search strategy the 
anomalous interactions for the fourth SM family quarks should be taken into account. These 
interactions may manifest themselves both in the production and decays of new quarks. If 
the anomalous decay modes of the fourth family quarks are dominant, lower limits on their 
masses given above are not valid and the search strategy should be changed drastically. 

In this letter, we consider pair production of the fourth SM family down-type quarks 
with subsequent anomalous decays into photon+jet and jet+jet channels. More general 
consideration, including other manifestations, will be presented in the following publication 
(23|. 

The effective Lagrangian for anomalous magnetic type interactions of the fourth family 



quarks is given as 



2J-|26|: 



L = Y,T 6qM4a ^ qiF " U + I^s^Vft^ + El^^ftGr + H.c 



(1) 



where F^ , Z^ v and G^ u are the field strength tensors of the gauge bosons, a^ u is the 
anti-symmetric tensor, T a are Gell-Mann matrices, e q is electric charge of quark, g e , gz 
and g s are electromagnetic, neutral weak and strong coupling constants, respectively, gz = 
g e /cos9y/sin9y/ where 9y/ is the Weinberg angle. k 7 , kz and n g are the strength of anomalous 
couplings with photon, Z boson and gluon, respectively. A is the cutoff scale for new physics. 
This type of gauge and Lorentz invariant effective Lagrangian have been proposed in the 
framework of composite models for interactions of excited fermions with ordinary fermions 



and gauge bosons (25 . 

For numerical calculations we implement the Lagrangian (1), as well as fourth family 



SM Lagrangian into the CalcHEP package 



27] . The partial decay widths of c? 4 for SM 



(dj —¥ W q where q = u, c, t) and anomalous (dj — > jq, dj —¥ Zq, dj — ¥ gq where q = d, s, 



b) modes are given below: 

TV J v W- \ l^4| 2 «eM| 

rK ~> w g) = i6Mp^;^^ (2) 

Wliere %VK — { 1 ~r ^q T %q%w ^q ^ w * x w)-> Xo = \± "T" 2^w "■" -^g 2X^ ^X^ 2X^X q ), 

x q = (M q /M d4 ) and x w = (M w /M d J, 

W -► Zg) = - a ' M * 2f) dfxzVZ (3) 

locos^wSzrrflw A 



2 „.2> 



where xz = (2 — x|— x|— 4x 2 — x 2 q x 2 z — Gx q x 2 z +2x q ), Xi — (l+x z +x 2 —2x 2 z — 2x 2 — 2x 2 z x 
and x z = (M z /M d4 ) 



r(4 - «) = ^(f )"* (4) 



where % 2 = (1 — 3s 2 + 3a;^ — x 



in 

T(d i ^ 1 q) = (-^) X 2 (5) 

One can wonder what is the criteria for the dominance of anomalous decay modes over SM 
ones. It is seen from Eq. (2)-(5) that the anomalous decay modes of the fourth SM family 
quarks are dominant, i.e. r(di —¥ gq) + T(di — >■ Zq) + r(di — > jq) > r(di — > W~q), if the 
relation («/A) > 1.2(1^ + V^ + V^J 1/2 TeV" 1 is satisfied (hereafter «| = «| = «« = « is 
assumed). The experimental upper bounds for the fourth family quark CKM matrix elements 
are \V U J < 0.063, \V U J < 0.46, \V U4b \ < 0.47, \V ud4 \ < 0.044, \V cdi \ < 0.46, [V^J < 0.47 
[28] . On the other hand, the predicted values of these matrix elements are expected to be 
rather small in the framework of flavor democracy hypothesis. For example, the mass matrix 
parametrization proposed in (29], which gives correct predictions for CKM and MNS mixing 
matrix elements through use of SM fermion mass values as input, predicts \V U4d \ = 0.0005, 
\V U4S \ = 0.0011, \V U4b \ = 0.0014, \V ud4 \ = 0.0002, \V cd4 \ = 0.0012, \V Ui \ = 0.0014. In 
this case, the anomalous decay modes are dominant, if (ft/A) > 0.0022 TeV -1 . The latter 
correspondens to upper limit 500 TeV for new physics scale A, assuming k = 0(1). 

The cross-section for the d^d^ pair production at the Tevatron is shown in Fig. 1. We 
have used CalcHEP |27| with CTEQ6L [30] parton distribution functions for numerical 
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Figure 1: Cross section for d^d^ pair production at the Tevatron. 



M d4 


200 GeV 


300 GeV 
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cuts 


as, fb 


(JB, fb 


as, fb 


as, fb 


a s , fb 


as, fb 


p T > 20GeV 


39.2 


5.4 x 10 5 


2.92 


5.4 x 10 5 


0.23 


5.4 x 10 5 


p T > 50GeV 


24.5 


2.7 x 10 3 


2.40 


2.7 x 10 3 


0.21 


2.7 x 10 3 


all cuts 


21.8 


3.63 


2.27 


0.091 


0.20 


0.006 



Table I: Signal and background cross sections values for various cuts. All cuts include p? > 50 
GeV, |t/| < 2, \M inv ( 7 j) - M di \ < 20 GeV, \M mv (jj) - M di \ < 20 GeV. 

calculations. One can see that if M di is about 400 GeV the Tevatron with L int = 10 fb -1 
will yield more than hundred d±d± pairs. 

We propose pp— > ^4^4—)- 'jqgq (where q — d,s, b) process to analyze the Tevatron search 
potential to discover d^ quark via anomalous decays. In detector this process is seen as 



7 + 3j events. We use CalcHEP 



27y and MADGRAPH |3l| packages with CTEQ6L (30 1 



parton distribution functions for the calculation of the signal and background processes, 
respectively. In order to extract the d^ signal and to suppress the background, following 
cuts are applied: p? > 50 GeV and \r]\ < 2 for all final state partons and photon, as well as 
invariant mass within ± 20 GeV around d^ mass. The effects of these cuts can be seen from 
Table I. 

Statistical significance has been calculated by using following formula 32] : 
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Figure 2: The necessary integrated luminosity for the observation of d/± quark at the Tevatron 
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5a discovery 
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Table II: Reachable M^ 4 mass values for discovery, observation and exclusion at the Tevatron. 



s 



2[{s + b)ln{l + 



(6) 



where s and b represents the numbers of signal and background events, respectively. 

In Fig. 2 the necessary integrated luminosities for the observation of d^ quark at the 
Tevatron are plotted as a function of d^ mass. It is seen that the fourth family down quarks 
with masses below 375 GeV could be excluded at 95% CL with 2.7 fb -1 integrated luminosity, 



if the corresponding ana, 
modes dominant case 



ysis of data has been performed (compare with 338 GeV for SM 
16]). Discovery, observation and exclusion for the different values of 
the Tevatron integrated luminosity are given in Table II. 

Summary and Outlook. - Keeping in mind the LHC status, the Tevatron has about 2 more 
years for new physics discovery. The fourth SM family quarks are among the most prominent 
candidates for beyond the SM3 physics and possible dominance of their anomalous decay 
modes should not be ignored. If these modes are dominant, the fourth SM family down 
quarks with masses up to 400-450 GeV can be observed (or excluded) via anomalous decays 



by Tevatron before the LHC. 
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